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Abstract — Guruswami and Indyk showed in [1] that 
Forney's error exponent can be achieved with linear 
coding complexity over binary symmetric channels. 
This paper extends this conclusion to general discrete- 
time memoryless channels and shows that Forney's 
and Blokh-Zyablov error exponents can be arbitrarily 
approached by one-level and multi-level concatenated 
codes with linear encoding/decoding complexity. The 
key result is a revision to Forney's general minimum 
distance decoding algorithm, which enables a low com- 
plexity integration of Guruswami-Indyk's outer codes 
into the concatenated coding schemes. 
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I. Introduction 

Consider communication over a discrete-time memory- 
less channel modeled by a conditional point mass function 
(PMF) or probability density function (PDF) Py\x(h\x), 
where x £ X and y £ Y are the input and output symbols, 
X and Y are the input and output alphabets, respectively. 
Let C be the Shannon capacity. Fano showed in [2] that 
the minimum error probability P e for block channel codes 
of rate R and length N is bounded by 

logP e 



lim 

N— >oc 



N 



> E(R), 



(1) 



where E(R) is a positive function of channel transition 
probabilities, known as the error exponent. For hnite 
input and output alphabets, without coding complexity 
constraint, the maximum achievable E(R) is given by 
Gallager in [3], 



E(R) =mzxE L (R,p x ) 



(2) 



where px is the input distribution, and E^{R,px) is given 
for different values of R as follows, 



max p >i {-pR+ E x (p,p x )} 

-R + E (l,p x ) 

max < p <! {-pR + E Q (p,px)} 



0<R<R x 

Rx 5: R < Rcrit 
Rcrit < R<C. 



(3) 



The definitions of other variables in ([3]) can be found in 
[4]. If we replace the PMF by PDF, the summations by 
integrals and the max operators by sup in @, the 
maximum achievable error exponent for continuous chan- 
nels, i.e., channels whose input and/or output alphabets 
are the set of real numbers [3], is still given by @. 
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In [4] , Forney proposed a one- level concatenated coding 
scheme, which can achieve the following error exponent, 
known as Forney's exponent, for any rate R < C with a 
complexity of 0(N 4 ). 



R 

EAR) = max (1 - r )E I — 
r.e[#,i] Vr, 



(4) 



where r a and R are the outer and the overall rates, respec- 
tively. Forney's coding scheme concatenates a maximum 
distance separable (MDS) outer error-correction code with 
well performed inner channel codes. To achieve E C (R), 
the decoder is required to exploit reliability information 
from the inner codes using a general minimum distance 
(GMD) decoding algorithm [4]. Forney's GMD algorithm 
essentially carries out outer code decoding, under various 
conditions, for 0(N) times. The overall decoding com- 
plexity of 0(N 4 ) is due to the fact that the outer code 
(which is a Reed-Solomon code) used in [4] has a de- 
coding complexity of 0(N 3 ). Forney's concatenated codes 
were generalized to multi-level concatenated codes, also 
known as the generalized concatenated codes, by Blokh 
and Zyablov in [5]. As the order of concatenation goes 
to infinity, the error exponent approaches the following 
Blokh-Zyablov bound (or Blokh-Zyablov error exponent) 
[5][6]. 



E {oo) (R) 



max 
PX ,r a e[§, l] 



*-R 

r a 



dx 



E L (x,px) 



(5) 

In [1] , Guruswami and Indyk proposed a family of linear- 
time encodable/decodable nearly MDS error-correction 
codes. By concatenating these codes (as outer codes) with 
fixed-lengthed binary inner codes, together with Justesen's 
GMD algorithm [7], Forney's error exponent was shown 
to be achievable over binary symmetric channels (BSCs) 
with a complexity of O(N) [1], i.e., linear in the codeword 
length. The number of outer code decodings required by 
Justesen's GMD algorithm is only a constant^, as opposed 
to O(N) in Forney's case [4]. Since each outer code decod- 
ing has a complexity of O(N), upper-bounding the number 
of outer code decodings by a constant is required for 
achieving the overall linear complexity. Because Justesen's 
GMD algorithm assumes binary channel outputs [7] [8], 
achievability of Forney's exponent was only proven for 
BSCs in [1, Theorem 8]. 

1 Strictly speaking, the required number of outer code decodings is 
linear in the inner codeword length, which is fixed at a reasonably 
large constant. 
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In this paper, we show that Forney's GMD algorithm 
can be revised to carry out outer code decoding for only 
a constant number of timefl With the help of the revised 
GMD algorithm, by using Guruswami-Indyk's outer codes 
with fixed-lengthed inner codes, one-level and multi-level 
concatenated codes can arbitrarily approach Forney's and 
Blokh-Zyablov exponents with linear complexity, over gen- 
eral discrete-time memory less channels. 

II. Revised GMD Algorithm and Its Impact on 
Concatenated Codes 

Consider one-level concatenated coding schemes. As- 
sume, for an arbitrarily small e\ > 0, we can construct 
a linear encodable/decodable outer error-correction code, 
with rate r Q and length N a , which can correct t symbol 
errors and d symbol erasures so long as 2t + d < N a (l — 
r Q — E\). Note that this is possible for large N Q as shown 
by Guruswami and Indyk in [1]. To simplify the notations, 
we assume N Q (1 — r — 6%) is an integer. The outer code is 
concatenated with suitable inner codes with rate Ri and 
fixed length iVj. The rate and length of the concatenated 
code are R = r Ri and N = N Ni, respectively. In 
Forney's GMD decoding, inner codes forward not only the 
estimates x m — [xi, . . . , Xi, . . . , xn ] but also a reliability 
vector a = [oil, . . . , 04, . . . , ajvj to the outer code, where 
x % G GF(q), < a t < 1 and 1 < i < N D . Let 

f 1 x = x 
-1 x 7^ x 

For any outer codeword x m = [x m i,x m 2, ■ ■ . , x m tj ], define 
a dot product follows 



s(x, x) 



(6) 



N 



/ ' < / ' M - ' / • - ' III I I ^ ^ 1 

i=l i=l 



OL ' X m — / ais(xi 7 Xrai) 



(7) 



Theorem 1: There is at most one codeword x m that 
satisfies 

ol ■ x m > N (r + £1). (8) 
Theorem Q] is implied by Theorem 3.1 in [4]. 
Rearrange the weights in ascending order of their values 
and let i\, , in be the indices such that 



Oi x < . . . < a ij < 



< CXi 



(9) 

Define q k = [q k (ai), . . .,q k (aj), . . . ,q k (a N J], for < k < 
l/£2, where e 2 > is a positive constant with l/e 2 being 
an integer, and gfe(«i J ) is given by 

if a i] < ke 2 
<?fc(a%) = { and ij < N a (l - r Q - Si) . (10) 

1 otherwise 



Define dot product q k ■ x m as 



No 

5 Xr, 



No 

q k ■ x m = 2_^1k[.oti)s(Xi,x mi ) = ^qk(ai)si. (11) 

Then following theorem gives the key result that enables 
the revision of Forney's GMD decoder. 

2 The revision can also be regarded as an extension to Justesen's 
GMD decoding given in [7]. 



Theorem 2: If a ■ x m > N a (f + (r + e x )(l - f)), 
then for some < k < l/e 2 , q k -x m > N {r + ex). 

Proof: Define a set of values Cj — (j — l/2)e 2 for 
1 < j < l/e 2 and an integer p — [oii N e )/£%], where 

l<P<l/e 2 -E 
Let 

A = ci 

A fc = c k+ i ~ c k ,l < k <p - 1 

\> = a ijV (l-r - ei ) + l _ C P 

Aft a */i-p + JV (l-i- - el ) + l _ a ih-p + No(l-r -£l) ' 

if p<h<p + N a (r 

o ~r £l) 

^p+N (ro+ei) — 1 — a i No ■ (12) 



We have 



k=0 



and 



1 < i < p 

3 -P+« o( i-. - ei ) P<3 <P + N {r + £l ) ' 

(13) 



P+iV (r +£i) 

k=a 



(14) 



Define a new weight vector ex. = [ai, . . . , 5i, . . . , &n ] 
with 

argmin^ ^j^lcj - a t \ a., < a lNo(1 __ ro _ ei) 



a., > a 



l « (l-r - ei ) 



(15) 

Define p k = [p k {a\), ■ ■ ■ ,Pk{a%), ■ ■ ■ ,Pk{aN )] with 1 < 
k < p + N (r + £1) such that for < k < p 



Pk = Qk, 

and for p < k < p + N (r + Si) 
at < a 



Pfc(«i) 



1 ai > a. 



H-p + Notl-ro-ej) 
k-p+Notl-ro-eO 



We have 



P+Noiro+E!) 

k=0 



(16) 



(17) 



(18) 



Define a set of indices 

U = {?i,? 2 , ■ ■ ■ ,ijv (l-r -si)}- (19) 
According to the definition of c^, for i ^ U, ai — ai. Hence 
at ■ x m = a ■ x m + (&j - ai) s,. (20) 



Since \a.i — «i| < £2/2, and s$ = ±1, we have 



^(5 < -a i )s 4 >-iV < ,(l-r -e 1 )^. (21) 



ieu 



3 Note that the value of p cannot be 0. Because if p = 0, 
i.e., cti N • 1 _ r = 0, then there are at least N a (l — r — 

£1) zeros in vector a. Consequently, a ■ x m < N (r + £1) < 
N + ifo (l — which contradicts the assumption 

that a ■ x m > N (^. + (r D + ei)(l - ^-j). 
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Consequently, a ■ x m > N D (4f + (r a + ex) (l - ^)) im- 
plies 

' X m >N (r + e 1 ). (22) 
If p k ■ x m < N (r + ex) for all p fc 's, then 

P+N (r +e 1 ) 

oc ■ x m = A fc p fc • a; TO 

fc=0 

p+iV (r +ei) 

< N (r + £l ) Xk 



k=0 



N (r +ex), 



(23) 



which contradicts ([22]) . Therefore, there must be some p k 
that satisfies 

p fe -x m > N (r +ex). (24) 

Since for k > p, p k has no more than A (r +£i) number 
of l's, which implies p k ■ x m < N (r + ex), the vectors 
that satisfy (j2"4")) must exist among p k with 1 < k < p. In 
words, for some fc, q fe • x m > A (r + £i). ■ 

Theorems [1] and [5] indicate that, if x TO is transmitted 
and a ■ x m > N a + (r Q + £i)(l — ^f)), for some < 
k < l/e2, errors-and-erasures decoding specified by q k 
(where symbols with q k (oti) — are erased) will output 
x m . Since the total number of q k vectors is upper bounded 
by a constant l/£2, the outer code carries out errors-and- 
erasures decoding only for a constant number of times. 
Consequently, a GMD decoding that carries out errors- 
and-erasures decoding for all q k s and compares their 
decoding outputs can recover x m with a complexity of 
0(N ). Since the inner code length iVj is fixed, the overall 
complexity is O(N). 

The following theorem gives an error probability bound 
for one-level concatenated codes with the revised GMD 
decoder. 

Theorem 3: Assume inner codes achieve Gallager's 
error exponent given in ((2|) . Let the reliability vector a be 
generated according to Forney's algorithm presented in [4, 
Section 4.2]. Let x m be the transmitted outer codeword. 
For large enough N, error probability of the one-level 
concatenated codes is upper bounded by 

Pe < P{a-X ro <iV (y + (r +£ 1 )(l-y))} 

< exp[-N(E e (R)-e)], ' (25) 

where E C {R) is Forney's error exponent given by (jlj and 
£ is a function of £i and £2 with e — > if £1, £2 — » 0. 

The proof of Theorem [3] can be obtained by first replac- 
ing Theorem 3.2 in [4] with Theorem[2j and then following 
Forney's analysis presented in [4, Section 4.2]. 

The difference between Forney's and the revised GMD 
decoding schemes lies in the definition of errors-and- 
erasures decodable vectors q k , the number of which deter- 
mines the decoding complexity. Forney's GMD decoding 
needs to carry out errors-and-erasures decoding for a 
number of times linear in N a , whereas ours for a constant 
number of times. Although the idea behind the revised 
GMD decoding is similar to Justesen's GMD algorithm 



[7], Justesen's work has focused on error-correction codes 
where inner codes forward Hamming distance information 
(in the form of an a vector) to the outer code. 

Applying the revised GMD algorithm to multi-level con- 
catenated codes [5] [6] is quite straightforward. Achievable 
error exponent of an m-level concatenated codes is given 
in the following Theorem. 

Theorem 4: For a discrete-time memoryless channel 
with capacity C, for any e > and any integer m > 0, one 
can construct a sequence of m-level concatenated codes 
whose encoding/decoding complexity is linear in N, and 
whose error probability is bounded by 

l0gPe >E^(R)-e, 



lim 

N^oo N 

E {m \R) = max 

px,r„e[£,l] 



YZx [el(( 



-,Px 



, (26) 

The proof of Theorem U can be obtained by combining 
Theorem |3]and the derivation of E {m \R) in [5] [6]. 

Note that lim^^ E^(R) = E^ (R), where 
E^(R) is the Blokh-Zyablov error exponent given in 
([5]). Theorem 0] implies that, for discrete-time memoryless 
channels, Blokh-Zyablov error exponent can be arbitrarily 
approached with linear encoding/decoding complexity. 

III. Conclusions 

We proposed a revised GMD decoding algorithm for 
concatenated codes over general discrete-time memoryless 
channels. By combining the GMD algorithm with Gu- 
ruswami and Indyk's error correction codes, we showed 
that Forney's and Blokh-Zyablov error exponents can be 
arbitrarily approached by one-level and multi-level con- 
catenated coding schemes, respectively, with linear encod- 
ing/decoding complexity. 
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